{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bayes Classifier with Hidden Markov Model emissions Coin Toss\n",
    "\n",
    "author: Nicholas Farn [<a href=\"sendto:nicholasfarn@gmail.com\">nicholasfarn@gmail.com</a>],\n",
    "        Jacob Schreiber [<a href=\"sendto:jmschreiber91@gmail.com\">jmschreiber91@gmail.com</a>]\n",
    "        \n",
    "In this example we will be using a simple Hidden Markov Model to demonstrate how it can be used in a Bayes classifier. In this particular example we have a couple friends who enjoy playing a game of chance. Someone tosses a coin and whenever it lands on heads, the tosser gains \\$1, otherwise he pays \\$1. However some of our friends are prone to using rigged coins in order to come out ahead in the long run.\n",
    "\n",
    "Players fall into 3 categories, non-cheaters, smart cheaters, and dumb cheaters. Non-cheaters, as the name implies, do not use rigged coins. Smart cheaters make sure to switch between rigged and unrigged coins in order to keep the deception going. Dumb cheaters only use rigged coins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "from pomegranate import *\n",
    "import numpy as np\n",
    "%pylab inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by creating our hmm's. Since this is a simple example, we will only have 2 states, rigged and unrigged, with discrete distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "rigged = State(DiscreteDistribution({'H': 0.8, 'T': 0.2}), name=\"rigged\")\n",
    "unrigged = State(DiscreteDistribution({'H': 0.5, 'T': 0.5}), name=\"unrigged\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create our non-cheater and dumb cheater since their hmm's consist of a single state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "non_cheater = HiddenMarkovModel(name=\"non-cheater\")\n",
    "non_cheater.add_state(unrigged)\n",
    "\n",
    "dumb_cheater = HiddenMarkovModel(name=\"dumb-cheater\")\n",
    "dumb_cheater.add_state(rigged)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now add our starting states to our models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "non_cheater.start = unrigged\n",
    "dumb_cheater.start = rigged"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also add the transitions to our non-cheater and dumb cheater."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "non_cheater.add_transition(unrigged, unrigged, 1)\n",
    "dumb_cheater.add_transition(rigged, rigged, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we bake our models to finalize the structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "non_cheater.bake()\n",
    "dumb_cheater.bake()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create our smart cheater."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "smart_cheater = HiddenMarkovModel(name=\"smart-cheater\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The structure of our smart cheater hmm should look like the following."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add in our starting transitions. Smart cheaters start out half the time with an unrigged coin, half the time without."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "smart_cheater.add_transition(smart_cheater.start, unrigged, 0.5)\n",
    "smart_cheater.add_transition(smart_cheater.start, rigged, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can add our transition matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "smart_cheater.add_transition(rigged, rigged, 0.5)\n",
    "smart_cheater.add_transition(rigged, unrigged, 0.5)\n",
    "smart_cheater.add_transition(unrigged, rigged, 0.5)\n",
    "smart_cheater.add_transition(unrigged, unrigged, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we can bake the model to finalize its structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "smart_cheater.bake()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view the structures of all our hmm's in the images below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABSCAYAAAD3qQfWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAf0klEQVR4nO2deXxU5b3/38/MZN8TSEjCkoIhIFRRUNwuIteiP2uL9oq2XgE33G5VvHit0vKr5ScuaEFvxbYuxXqltfV6a11uURBBJSjIogZBkbAkBBIy2WaSzGSW7++PM3OchEwWmMwE8rxfr/Oac57nnOd8z5nn+TzPeVYlImg0Go0mOlhibYBGo9EMJLToajQaTRTRoqvRaDRRRIuuRqPRRBEtuhqNRhNFtOhqNBpNFNGiq4kaSqkHlVIvx9qOnqKUEqXUKbG2Q3NyoUVXc1KgBVJzoqBFVzPgUUrZYm2DZuCgRXeAoJTap5S6Vyn1uVKqUSn1F6VUYoj/XKXUN0qpOqXUG0qpghA/UUrdppTarZSqV0otV0qpLu41Tim1OhBWtVJqQYh3vFLqJaWUQym1Qyk1KeS6AqXUa0qpI0qpvUqpu0L8zlZKbVRKNSilDimlnlZKxQf8Pgic9plSyqmUuibgfrlSanvgmlKl1Gkd3sfPlFKfA81dCO/FnT23Uup6pdQGpdSyQPjlSqnzAu4VSqkapdSckPu9qJR6Rin1j4CNG5RSQ5RSTwbC3qWUOqPrf1FzUiAiehsAG7AP2AQUANnATuC2gN80oBY4E0gAfgN8EHKtAG8BmcBw4AhwaZj7pAGHgPlAYuB4csDvQcAFXAZYgUeAjwN+FmAL8H+BeGAkUA5cEvCfCJwD2ICigP3zOth4SsjxmUANMDlwrzmBd5AQ8j62A8OApDDPEva5gesBL3BDIPyHgAPA8sA7nA44gNTA+S8G3vHEwHtZC+wFZodc/36s44ne+n6LuQF6i9IfbYjMdSHHS4DfBfZfAJaE+KUCHqAocCzABSH+fwXuD3OfnwDbwvg9CKwJOT4VaA3sTwYOdDj/AWBFmLDmAX8LOe4our8F/l+Ha74CLgx5Hzd2887CPndAdHeH+H03cH5eiJsdmBDYfxF4LsTvTmBnh+sbYh1P9Nb3m67LGlgcDtlvwSj1EvjdGvQQEadSyg4UYohTZ9emAiildgAjAu7/B6PkuKcXNiQGPu1HAAVKqYYQfyvwYeA+o4GlwCQgGaPEu6WL+4wA5iil7gxxi+fbZwao6OL6cPamhhxXh+y3AohIR7euzu/qXM1JihZdDUAV3wonSqkUIAc42N2FIjIu9FgpNRSjtNtbKoC9IlIcxv+3wDbgJyLiUErNA67qJrzFIrK4i3P0FHuaqKMb0jQAfwJuUEpNUEolAA8Dn4jIvmMI6y1giFJqnlIqQSmVppSa3IPrNgFNgcatJKWUVSk1Xil1VsA/DWgCnEqpMcDtHa6vxqgHDvIccJtSarIySFFKfV8plXYMz6TRRAwtuhpE5D1gIfAaRiPYKODHxxiWA/ge8AOMT/PdwEU9uM4XuGYCRgNTLfA8kBE45V7gWozGqeeAv3QI4kHgj4GeBFeLyKfAXOBpoB74BqMeVqOJKUpEf2FpNBpNtNAlXY1Go4kiWnQ1Go0mimjR1Wg0miiiRVej0WiiSHf9dHUrm0aj0fSesHOT6JKuRqPRRBEtuhqNRhNFtOhqNBpNFDkx516or4evvoJ9+6CqCg4cgIMHjf0vv4SGhm6D6JaUFMjMhEGDYNgwGDoUCgrgvPOguBiGDz/+e2g0XeDxeGhubsbj8eD1es3f4ObxeHC5XPh8PkQEr9cbNduUUlitVqxWK/Hx8cTFxWGz2bDZbO32bTYbKSkpJCQkYLEMjDKe1+vFZgsvrd2NSIt9Q5rTCdu2wZYtsHMnvP461NQYfjYb5OUZAlhYaGynngpZWYZgBn/j4yEtMOQ+K6t92B6Pse92Q0uLIeANDYawNzQY9zp4ECoqDFH/+mvj/JQUKCkxtrFj4fTTYdIkQ5g1mh7gdrtpaGjA5XLR1tZmbm63G4/Hg9/vN8/tTMzi4uJITEzEarWilMJms2GxWLBYLFitViwWSzu3UIKiGUpnou31ek1B9/v9+P1+083n8+Hz+Whra2uXIYTu+3w+Myyr1UpCQgLx8fHmlpCQQGZmJomJiV0K1YlEU1MT6enp4Sf573eiW1YG69bB5s3w6adGidbng8GDYdw4uOIKGDPGELthw6BDxOlzamsNm3btMn537jT29+wBEUN0b70VpkyBc8+FhITo2qfpN4gIra2tNDc309raisvlareJCBaLpVMhCu6npqYSFxcX60c5ZkQEp9N5VKbScR8MUU5KSiIxMdHckpOTyQotKJ0A+P1+LBZLPxfd/fvh3Xdh7Vp45RWjdHr22UbJceJE47e/f843NsLWrUZG8cwzRtVHUpJRHTFtGlx8sfEcA+QTa6AhIrS0tNDc3GyKbG1tLcH0lZiYSFJSEgkJCe1EJSMjo5uQT346y5BcLhetra14PB5sNhvJycntttTUVBL6d4GmH4pueTm8+Sa8+iqUln4rUL/8JUyeDCdw7g7AoUPw0UewZg2sWmVUWwweDJdeCs8/b1R5aE5YRISamhocDgdOpxOHw4Hf70cpRUJCAikpKeTk5JgC0fFTXtMzvF4vdrud5uZmM1NzuVwAZn1xfn4+qamppKSkxNjadvQT0a2qghdfhD/+0agbzc+HGTPgyith6tSTW4g+/9yoj379daMUfPXVcOONRole0+/xer00Njaam9PpBCA1NZW0tDTzNzk5mS7W7NREAK/Xi9PpNDO7mkAbT1xcHOnp6WRmZpKRkUFqamos/4sYi67fD2+8AVddZVQdXHcdXHONUaIdiJ/bv/41rFgBO3bA+PFw000wZ077Rj5NzKmtraWhoaGdyCYnJ5OZmUl6ejq5ublaYPsBPp/PFOBgpujxeLBaraSnp1NUVERaWlq0/6sYiW5bm1FH+8gjRsl2xQqYOdOoStAYPTJeegleftl4VzfeCPfeazQQamKCy+XCbrdjt9tpaGggKSmJjIwMsrKyyMzMPKEbtQYSLpfLFOBDhw5hs9nIysoiKyuLnJwc4vv+qzrKout2w29+A088YXS7mj3bEJPRo48puJOepiZ49llYtszoHbFnj9EvWBMVGhsbOXLkCHa7HZfLRVxcHFlZWRQXF5803ZgGMi0tLdTV1WG322lsbAQgPT2dnJwccnNz+6pBLkqie9FFsH49/Ou/GqVbLRzHxpIlsHgxpKcbPSF+8INYW3TS4Pf7qampoaqqira2NnJzc8nNzSU1VS/EO9AQEex2OzU1NdjtdnJycigsLIxUj5IoiO4LLxil22ef1Y1DkaCmBv7jP4zqh7lzYelS0MJwzLS1tVFVVUVVVRVer5fc3FzGjBkTa7M0/YTDhw9TVVWFw+EgLS2NoUOHMnjw4OOpB+5D0W1pMaoP/vY3Y79/95078XjtNbjtNqMB8u9/N0bcaXqM2+1m3759VFdXY7PZKCgooKCgIBp1epoTkKamJiorK6mtrSUuLo4RI0aQn59/LOLbR6J75Ijx6btnj9HfdurU3hqm6QmHDhkNkF9+aWRuF14Ya4tOCPbt20dFRQXx8fGMGDGC3NzcATP+X3N8uN1uKisrqaqqIiEhgbN7//XeB6JbWWnU4YrA//6vbiTra1wumDUL3nyTNW+9ZQ4PzcrKIj4+npycHJJ0rxCTmpoadu/ezfDhwyksLNRiqzkmXC4Xe/fuxe12U1JS0ps0FmHRbWoy5hbw+eC99yA3t6eGaI4Hvx/uuAP1+9936p2WlkZBQQGnnXYaw4YNo6SkhJKSEk499VQGDx4cZWNjg8/n45tvvuHw4cOcd955uouXJiJs3bqVlpYWiouLycvL68klERRdnw++/31jhNXHH/f/ORFiTGNjY2TH1/t81A8ZguP11/Hm51NfX4/H4+HIkSNmq/znn3/O/v37+eqrr2hqagIgOzubiRMnMmXKFO69914SExMjZ1M/wel08uWXX+L1eikpKSEnJyfWJkWVwOxWsTYjZkQ8rYUgIpSXl1NZWUlubi4lJSXdfT1FUHSXL4d//3fYuBHOPLPHRseKc845hylTprBkyZKo3dPtdvPrX/+at956i02bNkV+ntPTTjOmtFy9uttTq6qq2LVrF7t27WLjxo2sX7+eI0eOMG3aNGbPns2MGTPCCnBVVRXvvPMOq1atoqKigtLS0m7v94c//IFVq1aRkZHBtGnT+MlPftLrxzsWGhsbKSsrIzU1lbFjxw6ohrInnniCN998k9LSUjzBqUoHEA8//HDfpbUO1NXVsWvXLpKTkxk/fnxX/bjDt7yJSFdbe6qrRbKyRBYsOMqrv/LjH/9YFi5cGPX7tra2SnZ2thivOMJs2iRisYj85S/HdPmKFSvksssuE6vVKpmZmXL77bfL/v37Oz33wIEDAsiYMWO6DXfRokVSVFQk9fX1Ul9fL0VFRfLUU08dk429wel0yocffihlZWXi8/n6/H79DZfLJTk5OX0T10SkoqKiT8KNFH2a1jqhublZSktLZfv27eL3+8OdFlZXeye6Q4aILFoU6Wc4aRkzZkzfRYSyMuPv++ij4w7K7XbLE088IUlJSXLJJZeI0+ls598T0X344YcFkE2bNpluGzduFEAee+yx47YxHHv27JENGzaI2+3us3ucCPRVXBs3blyPMtxY06dpLQyHDh2S9evXi8Ph6Mw7rK72rkm3utoYbRZhnnvuOSwWC0opHA4HS5cuNY8B3njjDW699VaGDRvG9ddfz6BBg/jud7/Lli1b8Pv9rF+/nnvuuYfvfOc7TJ06lREjRlBXV8err77K9ddfz4UhXayefvppZs2axR133EFiYiJKKXPrif+SJUu4+eabOeuss/je975HWVmZGXZrayvz589n4cKFLFiwgObm5oi/K5Nx44zVKv70p+MOKj4+nvnz5/PRRx+xdetWbrrppl6H8dFHHwEwNGQU4rDAHBKfffbZcdvYGV6vl6qqKoYPHx7V6oRgfGxoaDgqPgZpamrigQceYP78+VxyySXMnz+fhsAyUl3F565obm7moYceYtasWdx9991M7aSL5pEjR7jqqqvIyclh/PjxprvL5Qobd3fv3s3MmTO5//77mT17NlOmTOGLL74AYMeOHRw+fJjbb789bFhlZWVHpcOqqiozLYYjGE5He0LfT0NDQ6fvJ5jWbr311r5Pa2EYMmQIKSkpVFRUtHPftm1b1xd2pchHaffw4X2RYYiIyKhRo9rlVKHHlZWVkpqaKoDs379fXn75ZQFk8uTJ4na7pbS0VJKTkwWQNWvWyM033yxOp7PTT2Or1Sp2u11ERB555BEBZP78+T3237Vrl7k/ffp0ycvLk6amJvF6vTJ58mSZO3euiBglMJvN1re57113iVxwQUSDfOedd46yueM77IwJEyYIIK2traZbS0uLAHLuuedG1MYgNTU1sn79evF4PH0SfjiC8XHx4sVHxUcREYfDIaNHj25n5+jRo2XkyJHS0NDQZXwOh8fjkalTp8qsWbPMT9oVK1bIm2++KSLflvR++ctfyr59++Ttt99u9z/OnTu307grIlJcXCyjRo0y75OZmSnjx48Xkc7/+45h5eXlSW1tbbt0+Mgjj5hpMRyh4YSmpdD3s3jx4qPeT0zSWhiqqqrkww8/bOe2ceNGkYhVL0Q4gYfS8fOg43FJSUm747y8PElISAjrH6RjpLFYLNLW1iYiImVlZQLIOeec0yP/Tz75RDAaF9ttb731ljz99NMCyM6dO82wRo8e3bcR4YknRIYNi3iwwQQXpCeiO2XKFAHE5XKZbq2trQLIxIkTI26jiMjevXtl8+bNfRJ2d5SUlLQ7Do2PP//5z4/631966SUB5L777jOv7yo+d2Tp0qUCyFdffWW6eb1eqa+vF5Fv00toHWNOTo6IhI+3wfsvXbpU/vznP4uIiN/vl1GjRklcXJyIHP3fd5UGQp+rrq6uy/fX03A6ez8xSWthcDqdsm7dunZu3Ylu76ZQiuHSIh2H4WVlZVFdXR3WPxx+v5+3336bK664wmy1nzZtWo/8N2/ebC6/0pEZM2YAUFRUZLr1eYf87GxjAc0Ik5+f3+trxowZwwcffEBDQ4PZj7E+YFtBHy3W6fV6Y7YiQ1fxccOGDUedP2XKFACzB0h38Xns2LHt/EcHBh+FVt9YrVYyMzPD2hXsm71582bGjx9vVhl05J577qG5uZlnnnmGuro6c2HMzugurOD9Q9c16/gsAD/96U97FE6Q0Pfz7rvvAlFOa2E4llnoendFVVWvb9DfeP7557npppvYsGEDu3fvZtGiRSxYsKBH/na7nZaWFpKTk9uF6ff7OXjwoHlOYWFhdB6mstJYATmCuFwutm/f3uvrxo0bBxjdzIKie+jQIQAuuOCCyBkYQnx8PG63u0/CPh46E4DgO+lpP9KdO3e2Oz7nnHMAo/719NNP75U9drud8vLyTuMuGEJ6zTXX8Mwzz3DHHXewcuXKXocVWIyxR88CsGjRol6HEyQmaS0MwaWDekPvsod9+4xRUX1AaM4mIua8l5GmrKyMjz/+mMcff5zXX3+dhQsXtistdeU/ZswYHnvssXbh7dy5k6efftqcsertt9/uE7s7Zc8e6KKhQkTYtWtXr4L81a9+Za7O2h2hy2vPmjWLzMxM3n//fdNt7dq1xMfHc+211/bKhp6SkZGB2+2OSSNKVwRLtaEEG1suvvjiYwozKLSLFy9u97X1j3/8o9trx4wZQ0tLS6dxF2D27Nl4PB4uvfRSgHZLvyul2vV97SysYBroDccTTkzSWhjq6up6P9Coq7qHoyowQGTt2j6pG7nyyisFkN27d8uyZcvMfnerVq0Sn88nRUVF7epsCgsLBTAbUTr6ixgNGoAUFBSYbqNGjZIXXnhBVq1aJaWlpfL111+L1+vtkb/L5RJAbrzxRlm5cqX84he/kOnTp0tTU5Ns375dbDab5OTkSEtLi6xdu1bS09MFkL1790b+hbW1iQwaJPLoo0d5tbS0yLPPPiujR4+WSZMm9Sg4j8cjCxYsEKWUrFixol1YgBQXF7c7f/HixZKVlSX79u0z3R577DEpLi4Wh8MhTU1NUlxcLIv6uIvhpk2b2tXtRYuioqJ2x6HxsaWlRcaPHy+HDh0y/e+++245//zzw8bXjvG5I+Xl5ZKSkiKATJs2TZYvXy4LFy4063ALCgoEkKamJvOa/Px8cTqd4nK5ZOTIkZ3GXRGRjIwMUUrJu+++KytXrpTc3FwB5JNPPpFTTjlFUlJS5MCBAyIinYYVTAOhz9Wx22FHQsPpmJa6ez+haW3VqlV9n9bC4PF4ZMOGDUfdM7INaWedJXLDDX3yAF9//bVMnjxZpk+fLl9//bX80z/9k8yaNUteeeUVWbZsmVnR3tjYKE8++aR5fNddd8mCBQvM423btomI0YH5gQceMN2XLl0qTU1NkpeXd1Tl/eDBg+W1114TEenW/4c//KFkZ2fLkCFD5JZbbpEjR46Yz/DBBx/I+eefLyNHjpRHH31UpkyZIrfddpu89957ke+0/8YbxgCJQGIIUlVVJZmZmWK1WkUpJUOHDu02qPXr18sZZ5whiYmJ8uKLL5ru77//vtxyyy0CSFxcnCxZskS2b98uIiLLli2TESNGSGVlZbuwXnjhBZk1a5bMnDlTnn322Qg8aNfU1NTIunXrum24iSTLly8XQB566KGj4uP9998vra2t4nA4ZPr06TJ//ny57777ZNGiRWZf4uD1ncXn4PWd8cUXX8gll1wiWVlZUlhYKPPmzRO/3y+PP/64ef28efPE6XTKkiVLzJ43brdb9u3bFzbuLl++XDIyMuTss8+Wjz/+WJ566inJysqSGTNmyAMPPCD5+flm/BeRo8I6cuSINDc3y6JFi0w7brnlFjMthiMYTmf2BMN56KGHOn0/wbSWlpbW92ktDLt27ZLS0tKjMsrIiq7PJ3L22SIXXigSfiRGv+XRRx81u4OFUl5eLmeccUa3/v2Gp54SiY8X344dsnr1arn00ktFKWV2m+m4BRO71+uVLVu2yGWXXSbx8fGSkJAgV199tZSWlsb4gY4Pn88nW7duldLS0m5LWBrN8VJRUSHr1q0zS/9hiJDoiohs3GiUsJYsidQzRI3LL79cFi9eLLW1tabbli1b5Oqrr5YdO3Z0699fcKSkyO8vu8zsy2y1WsN2CQJkzpw5ZqkAkB/96Efy0ksvSUNDQ6wfJWJ4vV757LPP5MMPP5SqqqpYm6M5CfH5fLJ79+6eCK5IF7p6bFM7LlsG8+fDypUQpQlNIkFdXR1nnXUWVVVVnHnmmRQWFjJ9+nTmzJlDXFxct/79gf2bNjHuvPNwK4XP5wvbhS2UCRMmcO655zJhwgTOO++8diOVTiZEhP3797N//34GDx7MqXqVDU2EaG5uZufOnbS1tfV0Brs+mMT8zjuNddFeeQV++MPuDNBEgI9ee427rruObS4XNputxzMqrV27losuuqiPres/BGeCOuWUU8jVcz1rjhOv18vGjRtJT0/vzQx2YUX32HsUP/mksZLBj35kTPeo6Vu2beOCO+9k6+jR/PWvf+XGG2805zawWq1hO2krpaitrY2mpTEnOzubSZMmsXPnTrZu3WrOeaDR9AYRobKykk8++YRRo0Zx+umnR2SOj94PpwhitcLvf2/0E73zTigrg9/+9rgN0nTCSy/Bv/0bnHsu/Pd/MzM9nZkzZwJQXl7OmjVrWL16NatXr6axsRGLxWL2tbTZbANOdMEYODFp0iTKy8v57LPPyMrKYujQoWRnZ8faNE0/x+v1cvjwYQ4ePEhbWxtDhw6N6KjKyCzB/uqrcOut8M47cNZZETFMA9jtxkrAr70G8+bBY49BF3XLfr+fbdu2cd9997FhwwZztNaiRYtYuHBhtKzud9TX11NRUUF9fT3JyckUFhaSl5cXsyHEmv5Ja2srBw8e5PDhw4Axi9iwYcNIOLYVzvtwCfYglZXwne/AXXfBwoXGkuGaY8PvN0q3CxYYXxQvvgj//M+9CsLtdlNaWsp7773H0KFDue222/rG1hOI5uZmDh48SHV1NRaLhfHjx/fZ8i6aEwefz8eOHTuor68nMTGRwsJChgwZckzzKoQQBdEF+N3vDMEFePBBo/R7fIYPTCZONNagmzsXFi+GkMlDNMePx+Ph8OHDlJeXk5CQQG5uLnl5eaSkpMTaNE2UEBHq6uqorq7GbreTnZ1NXl4eOTk5PZ48qxuiJLqhlJfD448bpbSMDKMEHDKxjCZAWxu8/rrxrj791CjRrlkTa6sGHE1NTdjtdux2O83NzVitVkaNGkVmZqZe2v4ERkQ4cOAAdrsdh8OBxWIhMzOT7OxsBg0adKxVBz0hBqIbpLoa/vM/jUa2qVPhppvg0kuNz+aBTFmZ0eVu5Uqoq4Mrr4Sf/QwmTYq1ZQMet9tNXV0d33zzDX6/n/j4eDIyMsjIyCAzM1OXiPsxPp+PpqYmGhsbaWxspKmpibi4OLKzs8nJySErKyta00DGUHSDtLYaS7evWwf5+fAv/2IIzZQpA0uAd+2C666DLVtg1Ci44QaYMwdC5knV9A9EBIfDYSbgxsZGvF4vNpuN9PR08vPzSUtL68vSkqYbRITq6mocDgcOhwOn04mIkJiYaGaSQ4YMiYVp/UB0g+zZA//1X0aLfFkZDBoEP/gBXH45XHghdD/S48SirQ0+/tjo2fG3v8HOnUZd7bXXGs8bmfojTRQQEZqbm80SVE1NDQBxcXGkpaWRmppKamoqSUlJJCcnx2xi7ZMVj8dDS0sLLS0tOJ1OHA6HOa1namoqaWlppKenk5GR0R8ywn4kuqF88w38z/8YdZqbNoGI0TXqoouMPqknqgBv3GiU6N9/HzZsgJYWGDkSZsyAK64wSveaEx6fz2cm/uBva2urMb5eKbKzs0lOTja3xMTEqC6geaIiIrjdblpbW2lpaaG2tpbm5mZzNQur1WpmcGlpaeTm5kaq8SuS9FPRDaWhAT74AH7xC6MELGJ0QZs0yej7O3EijB8P/W1Yp9MJX30FW7fC5s1GY9i2bUYVyrRpRgYybZrxLJqTHhExS2O1tbXmfnCwisViISkpicTERBITE8nMzDTFOC4urj+KR5/g9/tpa2vD7XbT0NCAy+UyN7fbbc4pEhcXx6BBg0hJSTEzr35Qiu0JJ4DohmK3GwIWFLFPP/12qaDJk2HsWCgpgaIioy506FAoKIC+KkWIGA2CBw9CaakhssEtuPxyaiqceaaRSdx+O5xySt/YojkhCRUVl8tFa2srbrf7qBVS4uPjiY+PJyEhgbi4OFJTU4mLi8Nms5m/wf3+ht/vx+v14vV68Xg85r7T6aStra3dFjpvSGgmlJiY2O64Pz5nd7jdbhISEk4w0e2MQ4eM+tC///1bwaushOCfpxTk5cGppxoDMzIzjf6tmZnGKK70dKPBLi3t277DDQ2GoAI0NhqDEvbuNdzr643fmhpD8INL2OTmQnExjBkDo0cb4j92rNEoNpAaBDURwe/343a72wlS6LHT6cTr9XY6m5zNZiMpKQmLxYLFYsFqtbb7tVgspmhZrdajStGhnf/9fn+7ZXpC3fx+Pz6fD5/PZ+4H3b1eL21tbXg8nqOuB6NkH8w4gplJMGMJzWBOJhwOB2lpaSeB6HaGz2eUQA8cMISxosIQ5lDRrK83zguumhsUV2gvwKmphjgXFbUX7MGDjcUfhw83StO6BKuJAT6fr13pMbjf2traThhDBVJEzHrQzmakC3VTSh01LDroFhTyzkTdZrOZVSPBUnhoqXwANyaepKKr0Wg0/ZM+mNpRo9FoNL1Gi65Go9FEES26Maa5uZmKigpzGsZo4HK5on5PjUZj0F2drkaj0WgiiC7pajQaTRTRoqvRaDRRRIuuRqPRRBEtuhqNRhNFtOhqNBpNFNGiq9FoNFHk/wP1xB/g4aW7LQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title(\"smart cheater hmm\")\n",
    "smart_cheater.plot()\n",
    "\n",
    "plt.title(\"dumb cheater hmm\")\n",
    "dumb_cheater.plot()\n",
    "\n",
    "plt.title(\"non-cheater hmm\")\n",
    "non_cheater.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can finally create our Naive Bayes classifier. We'll let 0 stand for a non-cheater, 1 stand for a smart cheater, and 2 stand for a dumb cheater."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "players = BayesClassifier([non_cheater, smart_cheater, dumb_cheater])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a sequence of coin tosses to test our classifier on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.array([list('HHHHHTHTHTTTTHHHTHHTTHHHHHTH'),\n",
    "                 list('HHHHHHHTHHHHTTHHHHHHHTTHHHHH'),\n",
    "                 list('THTHTHTHTHTHTTHHTHHHHTTHHHTT')])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now check the log probability of each string of data occurring under each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For sequence 1, 21.4% non-cheater, 68.0% smart cheater, 10.6% dumb cheater.\n",
      "For sequence 2, 0.173% non-cheater, 12.1% smart cheater, 87.7% dumb cheater.\n",
      "For sequence 3, 66.5% non-cheater, 33.0% smart cheater, 0.515% dumb cheater.\n"
     ]
    }
   ],
   "source": [
    "probs = players.predict_proba(data)\n",
    "\n",
    "for i in range(len(probs)):\n",
    "    print(\"For sequence {}, {:.3}% non-cheater, {:.3}% smart cheater, {:.3}% dumb cheater.\".format(i+1, 100*probs[i][0], 100*probs[i][1], 100*probs[i][2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we can classify whether a non-cheater (0), smart cheater (1), or dumb cheater (2) created the string of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequence 1 is a smart cheater\n",
      "Sequence 2 is a dumb cheater\n",
      "Sequence 3 is a non-cheater\n"
     ]
    }
   ],
   "source": [
    "output = players.predict(data)\n",
    "\n",
    "for i in range(len(output)):\n",
    "    print(\"Sequence {} is a {}\".format(i+1, \"non-cheater\" if output[i] == 0 else \"smart cheater\" if output[i] == 1 else \"dumb cheater\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also train our hmm's through Naive Bayes. Let's used the following data, all marked as known games for smart cheaters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.array([list('HHHHHTHTHTTTTH'),\n",
    "              list('HHTHHTTHHHHHTH')])\n",
    "\n",
    "y = np.array([1, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's fit our Bayes classifier to our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\n",
       "    \"class\" : \"BayesClassifier\",\n",
       "    \"models\" : [\n",
       "        {\n",
       "            \"class\" : \"HiddenMarkovModel\",\n",
       "            \"name\" : \"non-cheater\",\n",
       "            \"start\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : {\n",
       "                    \"class\" : \"Distribution\",\n",
       "                    \"dtype\" : \"str\",\n",
       "                    \"name\" : \"DiscreteDistribution\",\n",
       "                    \"parameters\" : [\n",
       "                        {\n",
       "                            \"H\" : 0.49218749999999994,\n",
       "                            \"T\" : 0.5078125000000001\n",
       "                        }\n",
       "                    ],\n",
       "                    \"frozen\" : false\n",
       "                },\n",
       "                \"name\" : \"unrigged\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"end\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : null,\n",
       "                \"name\" : \"non-cheater-end\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"states\" : [\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : {\n",
       "                        \"class\" : \"Distribution\",\n",
       "                        \"dtype\" : \"str\",\n",
       "                        \"name\" : \"DiscreteDistribution\",\n",
       "                        \"parameters\" : [\n",
       "                            {\n",
       "                                \"H\" : 0.49218749999999994,\n",
       "                                \"T\" : 0.5078125000000001\n",
       "                            }\n",
       "                        ],\n",
       "                        \"frozen\" : false\n",
       "                    },\n",
       "                    \"name\" : \"unrigged\",\n",
       "                    \"weight\" : 1.0\n",
       "                },\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : null,\n",
       "                    \"name\" : \"non-cheater-end\",\n",
       "                    \"weight\" : 1.0\n",
       "                }\n",
       "            ],\n",
       "            \"end_index\" : 1,\n",
       "            \"start_index\" : 0,\n",
       "            \"silent_index\" : 1,\n",
       "            \"edges\" : [\n",
       "                [\n",
       "                    0,\n",
       "                    0,\n",
       "                    1.0,\n",
       "                    1,\n",
       "                    null\n",
       "                ]\n",
       "            ],\n",
       "            \"distribution ties\" : []\n",
       "        },\n",
       "        {\n",
       "            \"class\" : \"HiddenMarkovModel\",\n",
       "            \"name\" : \"smart-cheater\",\n",
       "            \"start\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : null,\n",
       "                \"name\" : \"smart-cheater-start\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"end\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : null,\n",
       "                \"name\" : \"smart-cheater-end\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"states\" : [\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : {\n",
       "                        \"class\" : \"Distribution\",\n",
       "                        \"dtype\" : \"str\",\n",
       "                        \"name\" : \"DiscreteDistribution\",\n",
       "                        \"parameters\" : [\n",
       "                            {\n",
       "                                \"H\" : 0.7949526813880127,\n",
       "                                \"T\" : 0.2050473186119874\n",
       "                            }\n",
       "                        ],\n",
       "                        \"frozen\" : false\n",
       "                    },\n",
       "                    \"name\" : \"rigged\",\n",
       "                    \"weight\" : 1.0\n",
       "                },\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : {\n",
       "                        \"class\" : \"Distribution\",\n",
       "                        \"dtype\" : \"str\",\n",
       "                        \"name\" : \"DiscreteDistribution\",\n",
       "                        \"parameters\" : [\n",
       "                            {\n",
       "                                \"H\" : 0.49218749999999994,\n",
       "                                \"T\" : 0.5078125000000001\n",
       "                            }\n",
       "                        ],\n",
       "                        \"frozen\" : false\n",
       "                    },\n",
       "                    \"name\" : \"unrigged\",\n",
       "                    \"weight\" : 1.0\n",
       "                },\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : null,\n",
       "                    \"name\" : \"smart-cheater-start\",\n",
       "                    \"weight\" : 1.0\n",
       "                },\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : null,\n",
       "                    \"name\" : \"smart-cheater-end\",\n",
       "                    \"weight\" : 1.0\n",
       "                }\n",
       "            ],\n",
       "            \"end_index\" : 3,\n",
       "            \"start_index\" : 2,\n",
       "            \"silent_index\" : 2,\n",
       "            \"edges\" : [\n",
       "                [\n",
       "                    2,\n",
       "                    1,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ],\n",
       "                [\n",
       "                    2,\n",
       "                    0,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ],\n",
       "                [\n",
       "                    1,\n",
       "                    0,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ],\n",
       "                [\n",
       "                    1,\n",
       "                    1,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ],\n",
       "                [\n",
       "                    0,\n",
       "                    0,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ],\n",
       "                [\n",
       "                    0,\n",
       "                    1,\n",
       "                    0.5,\n",
       "                    0.5,\n",
       "                    null\n",
       "                ]\n",
       "            ],\n",
       "            \"distribution ties\" : []\n",
       "        },\n",
       "        {\n",
       "            \"class\" : \"HiddenMarkovModel\",\n",
       "            \"name\" : \"dumb-cheater\",\n",
       "            \"start\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : {\n",
       "                    \"class\" : \"Distribution\",\n",
       "                    \"dtype\" : \"str\",\n",
       "                    \"name\" : \"DiscreteDistribution\",\n",
       "                    \"parameters\" : [\n",
       "                        {\n",
       "                            \"H\" : 0.7949526813880127,\n",
       "                            \"T\" : 0.2050473186119874\n",
       "                        }\n",
       "                    ],\n",
       "                    \"frozen\" : false\n",
       "                },\n",
       "                \"name\" : \"rigged\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"end\" : {\n",
       "                \"class\" : \"State\",\n",
       "                \"distribution\" : null,\n",
       "                \"name\" : \"dumb-cheater-end\",\n",
       "                \"weight\" : 1.0\n",
       "            },\n",
       "            \"states\" : [\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : {\n",
       "                        \"class\" : \"Distribution\",\n",
       "                        \"dtype\" : \"str\",\n",
       "                        \"name\" : \"DiscreteDistribution\",\n",
       "                        \"parameters\" : [\n",
       "                            {\n",
       "                                \"H\" : 0.7949526813880127,\n",
       "                                \"T\" : 0.2050473186119874\n",
       "                            }\n",
       "                        ],\n",
       "                        \"frozen\" : false\n",
       "                    },\n",
       "                    \"name\" : \"rigged\",\n",
       "                    \"weight\" : 1.0\n",
       "                },\n",
       "                {\n",
       "                    \"class\" : \"State\",\n",
       "                    \"distribution\" : null,\n",
       "                    \"name\" : \"dumb-cheater-end\",\n",
       "                    \"weight\" : 1.0\n",
       "                }\n",
       "            ],\n",
       "            \"end_index\" : 1,\n",
       "            \"start_index\" : 0,\n",
       "            \"silent_index\" : 1,\n",
       "            \"edges\" : [\n",
       "                [\n",
       "                    0,\n",
       "                    0,\n",
       "                    1.0,\n",
       "                    1,\n",
       "                    null\n",
       "                ]\n",
       "            ],\n",
       "            \"distribution ties\" : []\n",
       "        }\n",
       "    ],\n",
       "    \"weights\" : [\n",
       "        -Infinity,\n",
       "        0.0,\n",
       "        -Infinity\n",
       "    ]\n",
       "}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players.fit(X, y)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
